Partial AUC Estimation and Regression

Accurate disease diagnosis is critical for health care. New diagnostic and screening tests must be evaluated for their abilities to discriminate disease from non-diseased states. The partial area under the ROC curve (partial AUC) is a measure of diagnostic test accuracy. We present an interpretation of the partial AUC that gives rise to a new non-parametric estimator. This estimator is more robust than existing estimators, which make parametric assumptions. We show that the robustness is gained with only a moderate loss in efficiency. We describe a regression modelling framework for making inference about covariate effects on the partial AUC. Such models can help refine an understanding of test accuracy. Model parameters can be estimated using binary regression methods. We use the regression framework to compare two Prostate-Specific Antigen biomarkers and to evaluate the dependence of biomarker accuracy on the time prior to clinical diagnosis of prostate cancer

Semi-parametric Regression for the Area Under the Receiver Operating Characteristic Curve

Medical advances continue to provide new and potentially better means for detecting disease. Such is true in cancer, for example, where biomarkers are sought for early detection and where improvements in imaging methods may pick up the initial functional and molecular changes associated with cancer development. In other binary classification tasks, computational algorithms such as Neural Networks, Support Vector Machines and Evolutionary Algorithms have been applied to areas as diverse as credit scoring, object recognition, and peptide-binding prediction. Before a classifier becomes an accepted technology, it must undergo rigorous evaluation to determine its ability to discriminate between states. Characterization of factors influencing classier performance is an important step in this process. Analysis of covariates may reveal sub-populations in which classifier performance is greatest or identify features of the classifier that improve accuracy. We develop regression methods for the non-parametric area under the ROC curve, a well-accepted summary measure of classifier accuracy. The estimating function generalizes standard approaches, and, interestingly, is related to the two-sample Mann-Whitney U-statistic. Implementation is straightforward as it is an adaptation of binary regression methods. Asymptotic theory is non-standard because the regressor variables are cross-correlated. Nevertheless, simulation studies show the method produces estimates with small bias and reasonable coverage probability. Application of the method to evaluate the covariate effects on a new device for diagnosing hearing impairment reveals that the device performs better in the more severely impaired subjects and that certain test parameters, which are adjustable by the device operator, are key to test performance

Regression analysis for the partial area under the ROC curve

Abstract: Performance evaluation of any classification method is fundamental to its acceptance in practice. Evaluation should consider the dependence of a classifier's accuracy on relevant covariates in addition to its overall accuracy. When developing a classifier with a continuous output that allocates units into one of two groups, receiver operating characteristic (ROC) curve analysis is appropriate. The partial area under the ROC curve (pAUC) is a summary measure of the ROC curve used to make statistical inference when only a region of the ROC space is of interest. We propose a new pAUC regression method to evaluate covariate effects on the diagnostic accuracy. We provide asymptotic distribution theory and inference procedures that allow for correlated observations. Graphical methods and goodness-of-fit statistics for model checking are also developed. Simulation studies demonstrate that the large-sample theory provides reasonable inference in small samples and the new estimator is considerably more efficient than the estimator proposed b

Simian hemorrhagic fever virus infection of rhesus macaques as a model of viral hemorrhagic fever: Clinical characterization and risk factors for severe disease

AbstractSimian Hemorrhagic Fever Virus (SHFV) has caused sporadic outbreaks of hemorrhagic fevers in macaques at primate research facilities. SHFV is a BSL-2 pathogen that has not been linked to human disease; as such, investigation of SHFV pathogenesis in non-human primates (NHPs) could serve as a model for hemorrhagic fever viruses such as Ebola, Marburg, and Lassa viruses. Here we describe the pathogenesis of SHFV in rhesus macaques inoculated with doses ranging from 50PFU to 500,000PFU. Disease severity was independent of dose with an overall mortality rate of 64% with signs of hemorrhagic fever and multiple organ system involvement. Analyses comparing survivors and non-survivors were performed to identify factors associated with survival revealing differences in the kinetics of viremia, immunosuppression, and regulation of hemostasis. Notable similarities between the pathogenesis of SHFV in NHPs and hemorrhagic fever viruses in humans suggest that SHFV may serve as a suitable model of BSL-4 pathogens

Bacterial loads measured by the Xpert MTB/RIF assay as markers of culture conversion and bacteriological cure in pulmonary TB

Introduction Biomarkers are needed to monitor tuberculosis (TB) treatment and predict treatment outcomes. We evaluated the Xpert MTB/RIF (Xpert) assay as a biomarker for TB treatment during and at the end of the 24 weeks therapy. METHODS: Sputum from 108 HIV-negative, culture-positive pulmonary TB patients was analyzed using Xpert at time points before and during anti-TB therapy. Results were compared against culture. Direct Xpert cycle-threshold (Ct), a change in the Ct (delta Ct), or a novel "percent closing of baseline Ct deficit" (percent closing) were evaluated as classifiers of same-day and end-of-treatment culture and therapeutic outcomes. RESULTS: Xpert was positive in 29/95 (30.5%) of subjects at week 24; and positive one year after treatment in 8/64 (12.5%) successfully-treated patients who remained free of tuberculosis. We identified a relationship between initial bacterial load measured by baseline Xpert Ct and time to culture conversion (hazard ratio 1.06, p = 0.0023), and to the likelihood of being among the 8 treatment failures at week 24 (AUC = 72.8%). Xpert Ct was even more strongly associated with culture conversion on the day the test was performed with AUCs 96.7%, 99.2%, 86.0% and 90.2%, at Day 7, Week 4, 8 and 24, respectively. Compared to baseline Ct measures alone, a combined measure of baseline Ct plus either Delta Ct or percent closing improved the classification of treatment failure status to a 75% sensitivity and 88.9% specificity. CONCLUSIONS: Genome loads measured by Xpert provide a potentially-useful biomarker for classifying same day culture status and predicting response to therapy